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Abstract 

We consider the residual empirical process in random design regres- 
sion with long memory errors. We establish its limiting behaviour, show- 
ing that its rates of convergence are different from the rates of conver- 
gence for to the empirical process based on (unobserved) errors. Also, we 
study a residual empirical process with estimated parameters. Its asymp- 
totic distribution can be used to construct Kolmogorov-Smirnov, Cramer- 
Smirnov-von Mises, or other goodness-of-fit tests. Theoretical results are 
justified by simulation studies. 

1 Introduction 

Consider a random design regression model, 

Yi = m(Xi) +£i, i = l,...,n, (1) 

where {X, X^i > 1} is a stationary sequence of random variables with a density 
/ = fx, independent of a centered, stationary long memory error sequence 
{e,Ei,— oo < i < oo}, with a distribution F £ and density f s . The goal of this 
paper is to study the asymptotic properties of the empirical process of residuals, 

n 

K n (x) := (!{*«<*} - F s (x)) , 

i=l 

where i.% = Yi — rh(Xi) and m(-) is an estimator of the function m(-). 



Residual-based inference is a standard tool in regression analysis. With this 
in mind, several authors considered empirical process of residuals in case of in- 
dependent random variables or weakly dependent stationary time series, see e.g. 
[3], [10], [1], [17], [5], [18], just to mention few. 

As for regression models with long memory errors, in [4], the authors ob- 
tained that in case of a parametric regression, m(x) — /3q + ftix, with a known 



1 



intercept, the limiting behaviour of K n (-) is similar to the limiting behaviour of 

a 

K n (x) := fasiKx} ~ Fe(x)) , 

i=l 

in the sense that a~\K n (-) and <j~\K n (-) converge weakly to, respectively, 
f s (x)Zi, where Z\ is standard normal and cr„,i is an appropriate scaling factor. 
However, if one considers a parametric regression when both slope and intercept 
are unknown, from the latter paper one can only conclude that 

a~\ sup \K n {x)\ = op(l). 

To see intuitively why this is the case, consider for a moment a simple model 
Yi = i-i+Si. Estimate /i by the sample mean Y and compute residuals £j = Yi—Y. 
Then 

K n (x) = K n (x + Y) + (F £ (x + Y)- F e {x)) . 

From a general theory for empirical processes based on long memory random 
variables e%, i > 1, we conclude 



a n \ sup 



op(l), 



K n (x)+f e (x)J2 £ i 

see e.g. [6], [8], [19]. On the other hand, using the Taylor's expansion, 

n 

(F e (x + Y)-F E (x))^f E (x)J2^- 



i=l 



Therefore, the contribution of e = X)"=i £ i cancels out and asymptotic behaviour 
of K n {x) cannot be concluded from that of K n (x). See [13] and [12] for precise 
results along these lines. 



The main goal of this paper is to establish a general theory on asymptotic 
behaviour for K n (-). In particular, this theory is applied to the parametric 
regression and a nonparametric regression; the latter in a longer, arxiv version 
of the paper. We will show in this paper, that convergence properties of K n {-) 
may be completely different from the asymptotics of K n {-). To do this, we will 
establish a second order expansion for K n (') (see Theorems 3.1 and 3.2). 

The established results can be used, in principle, to test whether the er- 
rors £i,...,£„ are consistent with a given distribution F e . If F e belongs to a 
one-parameter family {F e (-,8),9 € R}, then one needs to know the value of 
the parameter 8. Therefore, we discuss asymptotic properties of an empirical 
process of residuals, when a parameter 9 is estimated. The appropriate limit 
theorems are established in Section 3.4. Our theoretical results are confirmed 
by small simulation studies in Section 4. 
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The results for empirical processes in Sections 3.3 can be applied directly to 
establish limiting behaviour of quantiles (see [8, Section 5]). Furthermore, in 
a spirit of [8, Section 3], our results should be applicable to the error density 
estimation. However, a precise proof requires at least third order expansion 
of the residual-based empirical process (see Section 3.6). Finally, it would be 
interesting to establish corresponding results in case of fixed-design regression. 

2 Preliminaries: LRD error sequence 

In the sequel, Fjj(-), /[/(•) denote a distribution and a density, respectively, of 
a given random variable U . Also, if U has finite mean, we denote U* = U — ~E[U]. 

We shall consider the following assumption on the error sequence: 
(E) Si, i > 1, is an infinite order moving average 

oo 

Si = c fc f?i-fc, with c = 1, 

k=0 

where r]i, — oo < i < oo, is a sequence of centered i.i.d. random variables, 
independent of Xi, i > 1. We assume that E[e 4 ] < oo, E[ef] = 1, and for 
some a E € (0, 1), C& ~ fc~( aE+1 )/ 2 L (fc) as k — > oo, where L (-) is slowly 
varying at infinity. 

Let 

n n r 

£ n,r=Y^ T[ C i.Vi-3.- ( 2 ) 

i=l I . ■ ••• ; I s=l 

In particular, e n ,i = S™=i £ i an< ^ ^ m s < 

< r :=Var( en , r )~n 3 -'' a «^(n). (3) 
From [8] we know that for r < a~ x , as n —> oo, 

o--J.e njr 4 Z r , r = l,2, (4) 

where Z r is a random variable which can be represented by a multiple Wiener- Ito 
integral. In particular, Z\ is standard normal. Moreover, the random variables 
Zi,...,Z p are uncorrelated, see e.g. [11, Eq. (1.22)]. We also note that the 
convergence in (4) holds jointly. 
Furthermore, let 

n p 

S n , p (x) = J2 (l{e,<x } ~ Fe{x)) + j^i-ir^FtH^r- 
i—1 r—1 

Assume that F v (-) is 5 times differcntiable with bounded, continuous and intc- 
grable derivatives. We note in passing that these properties are transferable to 



3 



F s (-). Following [19, Theorem 3] and [8, Theorem 2.2] we conclude, in particular, 
that for a 6 < 1/2, 

^S n ,x{x) => fP(x)Z 2 , (5) 
where Z 2 is the same random variable as in (4). Otherwise, if a e > 1/2, then 

n- 1/2 £ n ,i(a;) W x [x), (6) 

where {Wi(x),x £ M} is a Gaussian process and =>■ denotes weak convergence 
in D[0, 1]. Furthermore, for a 6 > 1/3, 

a-isupl^^)! a 4o. (7) 

The structure of this Gaussian process and its covariancc is given in a rather 
complicated form; see [19] for more details. 

3 Results 

Let 

Aj := Ei - e t =Si- (Yt - m(Xi)) = m(JQ) - m(Xi), A = (A x , . . . , A„). 

3.1 Empirical process of residuals: a £ < 1/2 

The following result provides an uniform expansion of the process K n (-) and 
forms a basis for further analysis. 

Theorem 3.1. Assume (E) with a e < 1/2. Assume that F v (-) is 3 times 
differentiable with bounded, continuous and integrable derivatives. Suppose that 
A can be written as A 1 + (A i, . . . , A „), where 

6 n + ^ + ^ + ^^0; (8) 

°Yi,2 °n,2 °n,2 

• j-Ao A V, where V is a nondegenerate random variable; 

• |Aoi| = op{5 n ), uniformly in i. 
Then 



sup 

,,-eR 



K n (x) - K n {x) - f s (x) £ A, - \fP(*) A.? + fP(x)A e nA 

i=l i=l 

P (^-V n , 2 ) + op (d B <r ni i) + Op \J2 A i) ■ 
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In principle, this result is very similar to [4, Theorem 2.1]. However, we pro- 
vide op(-) rates of the approximation. This is crucial to establish limit theorems 
for the process K n (-). 

To have some intuition, let us write 

n n 

K n (x) - K n (x) = ^ 1 {e i <x+A i } - £ 1 {ti< x } 
i=l i=l 

n n 

= ]T (ife^x+Aj - F e (x + A,)) - £ (M^} - Fe( x )) 

i=l i=l 
n 

+ ^(F e (x + A i )-F e (x)). (9) 
i=i 

From Theorem 3.1 and (7) we conclude for a e < 1/2 that, uniformly in x, 

n . n 

K n {x) = K n {x) + f e (x) £ A,: + -f'M E A ? " (*)Aoen,i 

i=l i=l 

+o P (<r n>2 + S n a n ,i) + Op I £ A| 



n n 1 n 

= -/ e (!c) + ]T A, + /W^en.a + -/«(*) £ A? - ./f^Ao^! 

i=l 2—1 i—1 

+ P ff>^. (10) 

We note in passing that in order to obtain the above expansion via (7) one has 
to assume that F v (-) is 5 times differentiable. 

As we will see below (Section 3.3), it may happen that the first order con- 
tribution 

n n 

-/ e (*)5> + /«(*)£ 

i=l i=l 

is negligible. In other words, the rates of convergence of K n {-) will be different 
from those for K n (-). The rates of convergence will be determined by the second 
order term 

1 n 

/f'(iV2 + -f^(x)J2 A > - /J^AoEn,!. 

i=l 

3.2 Empirical process of residuals: a £ > 1/2 

Let £j = £i — rji. Define £, n . r in the analogous way as e n y, see (2). 
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Theorem 3.2. Assume (E) with a £ > 1/2. Assume that F n (-) is 3 times 
differ entiable with bounded, continuous and integrable derivatives. Suppose that 
A can be written as Aol + (Aqi, . . . , Ao„), where 

S„ + V^Sl ->■ 0; (11) 

• j-Ao A V, where V is a nondegenerate random variable; 

• |Aoj| = op{5 n ), uniformly in i. 
Then 



K n (x) = K n (x) + f s (x)Y / ^+[fe(x)^2A i -Ux)Y,Si-fP(x)A ^ nil 

+ + Op(V^) + Op (<Wl) 

where n -1 / 2 (K n (x) + f e (x) Ym=i e converges weakly to W\(x) from (6). 
3.3 Application to parametric regression 

The results of Theorems 3.1 and 3.2 are the tools to establish a limit theorem 
for K n {-) in case of parametric model 

m(x)=0o+0ix- (12) 

We assume that the regression parameters arc estimated using standard least 
squares. We make the following assumption on the predictors Xi, i > 1: 

(P) Xi, i > 1, is a random sequence such that sup,jE[|Xj| + \X\] < oo. 

Corollary 3.3. Assume (P) and (E) and that 

(h-(3i=o P (a nA /n). (13) 

Assume that F v (-) is 5 times differentiable with bounded, continuous and inte- 
grable derivatives. 



(a) If a e < 1/2, then 



0~n,2 



where Z\ , Z2 are defined in (4) . 
(b) Ifa e > 1/2, then n^K^x) Wx{x) 
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Remark 3.4. Note that the rate of convergence a n .\ for the original process 
K n {-) changes to a n ,2 or y/n for K n (-)- The similar phenomena was observed 
in a context of empirical processes with estimated parameters in [13] (see also 
[2]). Note further that a possible LRD of predictors does not play any role. 

Furthermore, from the proof of Corollary 3.3 below, we may conclude that 
in case /?o = the limiting behaviour of K n (x) and K n (x) is the same. In other 
words, for the model (12) with /3q = 0, we have (see also [4]) 

a-^K n (x) => f E (x)Z 1 . 

Remark 3.5. The condition (13) can be verified for many stationary sequences. 
In particular, if Xi, i > 1, is LRD linear sequence with parameter ax, then the 
rate of convergence of (/3i — (3\) is either yfn or n ( a x+a E )/2 ^ £ or ax _|_ q, £ > i or 
max(ax,a E ) < 1/2, respectively; see [1G] and [7]. 

Proof of Corollary 3.3. Least squares estimation leads to the following ex- 
pressions: 

= — \-Y'X j e j -Xe) , &>-Po = e-X0i-Pi), (14) 
s„ \ n. =1 J 

where X and e are sample means based on X±, . . . , X n and £%,..., e n , respec- 
tively, and s n = i Y%=i(Xj — X) 2 . We have 

A, = m(JSQ) - m(X,) = (/3 - /3„) + (ft - ft)J5Q = e + (ft - ft)(Xi - X). 

(15) 

From (3) we conclude that 

e = Op(a n .i/n), ^ n ,xl n ~ a n,2, as n ~> oo. (16) 

From (13) and Assumption (P) we conclude Aj = e+op(a nt i/n)Op(l). Let now 
5 n = Cn.i/n. It is straightforward to check that such S n fulfills (8). Therefore, 
the conditions of Theorem 3.1 arc fulfilled with Ao = e and V = Z\. 

Furthermore, from (15), J2i=i = ne = e n .i = $^ i=1 £ i an( i vl& (16), 

n 

2J A^ = ne 2 + neo P (a nA /n) + op^a^^/n 2 ) = ne 2 + o P (a n , 2 ). (17) 
i=i 

Consequently, noting that S n a n ^i ~ a nj 2 and ne 2 — ee n ,i, the expansion (10) 
reads 

K n (x) = fP(x)e n , 2 - ^fP(x)ne 2 + o P {a n , 2 ) =: S n (x) + o P {a n , 2 ),(l8) 
uniformly in x. The result of part (a) follows now from (4). 

As for part (b), we recall that Y17=i ^» ~ SiLi £ i = 0- Also, since a E > 
1/2, A £„,i = <M<i/») = OpK, 2 ) = op(v^) and via (17), £™ =1 A 2 = 
Op(a n , 2 ) = op{y/n). Finally, the choice of S n yields d n a„.i = op{*Jri). There- 
fore, part (b) follows from Theorem 3.2. □ 
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3.4 Residual empirical process with estimated parameters 

Let us focus on the parametric regression model of Section 3.3. From Corollary 
3.3 

— sup \K n {x)\ A S up\fP(x)\(Z 2 - \z\\ 

for a e < 1/2. The above result can be used, in principle, to test whether 
the errors s%, . . . ,e n are consistent with a given distribution F £ . If however F £ 
belongs to, say, a one-parameter family {F e (-,9),9 € K}, then one needs to 
know the value of the parameter 9. A straightforward procedure would be to 
estimate it and use the statistic 



sup | E l{e i<x } ~ Fe(x;0 n )\, 

where F e (x;9 n ) is the distribution function F e (x) = F e (x\0) in which the pa- 
rameter 9 has been replaced with its estimator 9 n . 

Therefore, this section is devoted to study the limiting behaviour of 

n 

L n {x) := ^2 (^{ii^x} ~ F e( x 'i ■ 
i=l 

The results below may be seen as counterpart to the asymptotic results for 

n 

L„(x) := E y-iei^x} ~ F e (x;§ n )J , 

i=l 

see [13] for results and references therein for more discussion on this approach. 

Many estimators 6 n of 9 can be obtained with help of partial sums JZILi 
where H is a function that does not depend on n. Let us note that from Theorem 
3.1 we may have two scenarios for a e < 1/2: 

(A) cr~ 2 (SiLi — E[H (£j)]) converges in distribution to a nondegenerate 
random variable; 

(B) < 2 (Eti H(ii) - E[ff( £i )]) = o P (l). 

Example 3.6. Consider H(u) = u 2 which yields the estimator of Var(e). We 
obtain for a e < 1/2: 

) (zl- 1 -zi]=2(zt-h 



< 2 f^H^O-E^eOn 4- J fW(v)dH(v 
Consider now H{u) — u 3 . We have for a E < 1/2: 



2 



^i 2 - \zl 
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Consequently, if f £ is symmetric, then the right hand side is simply and thus 
we are in scenario (B). 

In what follows, we will write f £ (-;8) to indicate the density with the true 
parameter 6. 

Corollary 3.7. Assume that 6 n = i Yh=i and 9 = ^l H ( £ )}- Under the 

conditions of Corollary 3.3, we have 

-i-i„(x) => (fP(x;6)+f e (x;6) J /« (u;8)dH(u^j (z 2 - ±Z?j , 
n-^ 2 L n (x) ^ W^x) + f E (x;6) [ Wi{u) dH(u), 



respectively for a £ < 1/2 and a £ > 1/2, provided that the integrals at the right 
hand sides are finite. 

Remark 3.8. In case a £ > 1/2, one needs very restrictive conditions on finite- 
ness of J W\{u) dH(u). In principle, it requires that H has a finite support. 

Remark 3.9. We note that rates of convergence for L n (-), residual empirical 
process with estimated parameters, are the same as for K n {-), the ordinary 
residual empirical process. This is different as compared to K n (-) and its "esti- 
mated" version; see [13]. 

Proof of Corollary 3.7. We conduct the proof for a £ < 1/2. For a function 
g(x; 8) denote by Vgg(x; 8) its rth order derivative with respect to 8, evaluated 
at 8 = 8. In particular, V = V 1 . Then 

L n (x) = K n (x) + n(8 - 8 n )V e F £ (x; 8) + ^n(8 - 8 n ) 2 V 2 g F £ (x- 8* n ) 

with some 8* such that |0* — 8 n \ < \8 — (9*|. Therefore 



L n (x) = K n (x) + f £ (x; 6) ^(E[H(e)] - fT(e<))J + op(<r„, 2 ) 
= K n (x) - f £ (x; 9) (j H(u)dK n (ufj + o P (a n , 2 ) 
= K n (x) + f £ (x; 9)( I K n {u)dH(u)) + o P (<T n , 2 ) 



and the result follows from Corollary 3.3. 
3.5 Nonparametric regression 

Now, wc establish the result for nonparametric regression case. It is assumed 
that m(-) is estimated by the usual Nadaraya- Watson estimator, i.e. 

1 " 

m(x) = m b (x) = —j— J2 Y i K ^ x - X i)> ( 19 ) 
nbfb{x) j=1 
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with 

1 - 

h{x) = -^Y,K h (x-X j ), (20) 

where -ftTb(-) = K(-/b) and A'(-) is a positive kernel, which fulfills standard 
conditions: J K(u) du = 1, J uK(u) du — and J u 2 K(u) du < oo. 
Here we shall assume for simplicity that 

(PI) Predictors arc i.i.d. 

Results can be extended to LRD stationary predictors using estimates from [15]. 

Corollary 3.10. Assume (PI) and (E). Assume that F n (-) is 5 times differen- 
tiable with bounded, continuous and integrable derivatives. Also, suppose that 
the bandwidth fulfills 

b+{nb)- x ^Q, (21) 
6o* i/n ->■ oo. (22) 

(a) If a s < 1/2, and 

b 2 n/a rh i + n6 4 /cr„ :2 + b 2 a n:1 /a n , 2 + 2 " a , -> 0. (23) 

a n,2 nt) 

then 

—K n (x)^fP(x)(Z 2 --Z*), 

0~n,2 2 

where Z\ , Z 2 are defined in (4) . 

(b) Ifa E > 1/2, and 

b 2 n/a nA +nb 4 /Vn+b 2 a nA /^/ri + al >1 /(n 2 b) -> 0. (24) 

tften Ji _x / 2 J^„(a;) Wi(a:) . 

Remark 3.11. The condition (21) is standard in nonparametric estimation. 
With the standard bandwidth choice b = Cn~ x / 5 (see e.g. [14]) condition (22) 
is valid for a £ < 4/5. Likewise, one can easily verify that (23) holds for a £ < 4/5 
as well and so for all a £ < 1/2. Finally, (24) holds for 1/5 < a £ < 4/5 and so 
for all 1/2 < a £ < 4/5. 

Proof of Corollary 3.10. In the nonparametric regression model we have 

1 " 

A, = m fe (JQ) - m(Xi) = ifc(*<) + -j— - ^K^Xi - Xj)e jt (25) 

nbf b {Xi) j=1 

where 

1 " 

Rb ^ = "TTTT E( m ^) - m (v)) K b (V ~ *i) ■ ( 26 ) 
nbfb{y) j=1 
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Denote p(y) = (m/)"(y) - m(y)f"(y). Uniformly over {y : f(y) > 0}, 

R b (y)- b -^^ = 0(b\l + o P (l))). (27) 

Now, in the second part of (25), we may replace fb(Xi) with f(Xi). This is 
allowed since, first, /&(•) is the consistent estimator of /(•); second, since K(-) 
has bounded support 1 and f(x) > 0, x € I. Define for j ^ i, 

L b (X i ,X j ) = -^j-K b (X i -X j ). 

We may write (recall that L'l(Xi, Xj) is the centered version of L b (Xi,Xj)) 

1 ™ 

A, = R b (Xi) + E{Lb(X 1 ,X 2 )}e + - ^LKXi^^sj. 

71 3=1 

Using (27) and (52) below we argue that 

A, = Op(b 2 ) + e + o P {a ntl /n), (28) 

uniformly in i, provided that (22) holds. Therefore, the conditions of Theorem 
3.1 are fulfilled with Ao = e, S n = <J n .i/ n an d V = Zi, as long as (22) and the 
first part of (23) hold. 

From (28), 

n 

A 2 = ne 2 + P (b 2 a nil ) + P (nb 4 ) + o P (a^ 1 /n) = ne 2 + o P (a n>2 ), 
if a e < 1/2 and (23) holds. Likewise, if (24) holds and a e > 1/2, 

n 

]T A 2 = ne 2 + op( v / n) = P (a 2 nl ln) + o P (y/n) = o P (y/n). 

i=l 

Also, from Section 5.5 we obtain 

n n 

J2 A 2 - £l + °M<r»,2 V >/n)- (29) 

i=l i=l 

This finishes the proof. □ 

3.6 Conjecture on error density estimation 

We consider again the parametric regression model Yi = (3q + fi\X\ + e». Our 
goal is to estimate the error density f e . We use the standard Parzen- Rosenblatt 
estimator 

1 " 

fh,A(x) = -rY,K h (x-ii). (30) 

i=i 
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Conjecture 3.12. Assume (P) and (E) and that (13) holds. Furthermore, as- 
sume that F^-) is 5 times differentiable with bounded, continuous and integrable 
derivatives. If a £ < 1/2 and 

nh 5 -> 0, a,i,2^ -> oo. (31) 

then 

— (fhMx) - /(*)) A ff\x){Z 2 - \z\). 

Remark 3.13. The first part of (31) is the standard condition which assures 
that a bias is negligible. As for the second part, note that 



JL(/ fc ( a; )_/( a: ))4/(i)( a; ) Zlj 



where is the Parzen-Rosenblatt estimator of f e based on E\, . . . ,e n . The 
above result is valid if a n \h — > oo. In other words, <J n ^h — > oo is a large 
bandwidth condition which assures that the estimator has LRD-type behaviour. 
Otherwise, if a n .2h — > is should be expected that the rate of convergence is 
\fnh. However, the methods of this paper are not applicable to such situation. 
The same applies to the case a s > 1/2. 

The reader is referred to [20] and [15, Section 3.2] for precise results on kernel 
density estimation under long memory. 

"Proof". Clearly 

n (fh,A(x) - E[/ 8 (x)]) = \j K f^p) dK n (x - vh). 

Using (18), and integrating by parts we write the left hand side as 
1 / „, i .m, , , 1 



£ n ,2 - ^ne 2 j j K(v)fP(x-vh)dv+— j (k n (x - vh) - S„ (x - wh)) dK{v) 
Therefore, we expect 

— n (f hA (x) - E[f s (x)]) A fi l \x){Z 2 - \zl). 
This, however, requires a more precise op bound in (18). □ 



4 Simulation studies 

We conducted simulations justifying our results on asymptotic behaviour of 
supremum of the empirical process of residuals K n {-). First, we simulated 
n = 100 i.i.d. random variables i = l,...,n from N(0, 1) distribution. 
Then, supremum sup xgH K n (x) was calculated. This procedure was repeated 
100 times. Quartiles and standard deviation of the empirical distribution of the 
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supremum was calculated. Next, for the same errors, model Yi = 1 + iXi + 
was considered, and residuals were calculated using estimators of j3o, Pi given 
in (14). Also, for the same errors, we assumed that /3q = 1 is known. The 
same procedure was repeated with errors following LRD Gaussian process with 
a e e {0.2, 0.4, 0.6, 0.8}. The results are given in Table 1. 

• Column 3: For the empirical process K n based on errors, the variability of 
the supremum increases with the dependence, which is in agreement with 
the asymptotic theory for the LRD-based empirical processes. 

• Column 4: We consider the empirical process L n , where F £ (-) is re- 
placed with F e (-; 9 n ), 9 n being sample standard deviation based on errors 
£\, . . . ,e„. The results are similar to column 3. In other words, estima- 
tion of variance does not influence asymptotic behaviour of the empirical 
process. This agrees with theoretical results; see [13, Remark 1.6]. This 
happens since variance can be estimated with rate cr„,2 V y/n, whereas the 
rate of convergence for K n (-) is <J n ,i- 

• Column 5: We consider the residual-based empirical process K n in the 
linear regression model. Both slope and intercept are estimated. We note 
that the variability for a £ = 0.8 or a e = 0.6 is almost the same as for i.i.d. 
case. In other words, LRD does not play any role, which is in agreement 
with Corollary 3.3. 

• Column 6: Results for the residual-based empirical process L n with esti- 
mated variance are similar as for K n . Recall that Corollary 3.7 indicates 
that rates of convergence for L n is the same as for K n . 

• Column 7: We consider K n , but the intercept is assumed to be known. 
Results are similar to Column 3. In other words, in case of known intercept 
the asymptotic behaviour of K n is similar to K n ; see Remark 3.4. 



5 Technical details 

Let Hi = a(rji, rji-i, . . .). Let u = (ui, . . . , u n ) be a vector of scalars. Define 

n n 

Z„(x; u) = ^2 ^{a^x+ui} - F £ (x + m)) - ^2 (l{ £i < x } - F s(x)) ■ 

i=l i=l 

The process Z n (x\ u) is written as 

n 

Z n (x; u) = ^2 i)-{x<ei<x+ui} - E [l{ x<Ei < x+Ui y\Hi-i] ) (32) 

i=l 

n 

+E( E C 1 

\Hi-i] - E [l{ !C <e i <s+« i }]) =: M n (x; u) + N n (x; u). 

i=l 
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K n 


L n 


K n 


L n 


Kn, A) = 1 


L n \ A) = 1 




Qi 


0.0416 


0.0392 


0.0467 


0.0413 


0.0419 


0.0376 


i.i.d. 




0.0880 


0.0859 


0.0656 


0.0592 


0.0873 


0.0789 




s 


0.0314 


0.0315 


0.0169 


0.0146 


0.0312 


0.0313 




Qi 


0.0307 


0.0274 


0.0473 


0.0448 


0.0346 


0.0278 


a e = 0.8 




0.0994 


0.0940 


0.0686 


0.0637 


0.0963 


0.0908 




s 


0.0484 


0.0494 


0.0149 


0.013 


0.0494 


0.0504 




Qi 


0.0303 


0.0150 


0.0488 


0.0447 


0.0274 


0.0147 


a e = 0.6 


Q 3 


0.1285 


0.1237 


0.0718 


0.0646 


0.1303 


0.1192 




s 


0.0758 


0.0786 


0.0151 


0.0139 


0.0772 


0.0797 




Qi 


0.0062 


0.0038 


0.0504 


0.0471 


0.0072 


0.0041 


a E = 0.4 


Q 3 


0.1471 


0.1353 


0.0784 


0.0662 


0.1479 


0.1353 




s 


0.0858 


0.0852 


0.0194 


0.0147 


0.0850 


0.0845 




Qi 


0.0015 


0.0023 


0.0535 


0.0418 


0.0021 


0.0017 


a E = 0.2 


Q 3 


0.2870 


0.2714 


0.0826 


0.0645 


0.2978 


0.2770 




s 


0.1911 


0.1851 


0.0218 


0.0178 


0.1906 


0.1850 



Tabic 1: Simulated values of different dispersion measures. 



Recall now that A = (Ai, . . . , A n ). Recalling (9), we decompose 

K n {x)-K n (x)= (33) 
= M n (x; A) + N n (x; A) + f s (x) £ A, + -fP(x) £ A? + O ^ A| 

»=1 i=l \»=1 

First, in Corollary 5.2 we will establish an asymptotic expansion for the LRD 
part N n (x; A). This will be done by considering a special structure of N n (x; u) 
(see Lemma 5.1 and (35) below) and then "replacing" u with A under proper 
assumptions for the latter. 

Furthermore, we have to bound M n (x; A). This will be done by obtaining 
an uniform bound on M n (x;u). In this way, we may utilize the martingale 
structure of the latter. Clearly, M n (x; A) is not a martingale. The bounds are 
given in Lemma 5.3 and Lemma 5.5. 

5.1 LRD part 

Denote Uo = UqI, where 1 is the vector of dimension n, consisting of T'. Recall 
that & = £i — rji and £ n>r is defined in the analogous way as e n , r - 

In the first lemma we deal with N n (x; u ). The proof is included in Section 
5.1.1. 

Lemma 5.1. Assume that Fn(-) is 3 times differ entiable with bounded, contin- 
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uous and integrable derivatives. Then with some < v < 1/2 and S n — > 0, 

sup sup N n {x- u ) + f^(x)u Ui = Op (<£-" (a n . 2 V ^) + S 2 n ^a n A ) . 

|«ol<^"" xm 

(34) 

Note now that the part N n (x,u) in (32) can be written as 

n 

N n (x; u)=^2 (F v ( x + u i - &) - F ri {x - &) - EF v (x + Ui - &) + EF„(x - &)) ■ 

i=i 

Let us choose u = Uo + (itoi, • • • , uon)- If maxj(|uoi|) = o(S n ), then applying 
first order Taylor expansion, and noting that £j, i > • 1, is LRD moving average 
with the same properties as i > 1, 

n 

N n (x; u)-N n (x; u ) = o(5 n ) ^ (fn( x + u o - &) - E/^rc + u - &)) = 0p(<Wn,i)> 

i=l 

uniformly in u, Uo and x, since is bounded and integrable. Combining this 
with (34), we have (recall v < 1/2) 



sup sup 

u i£l 



N n {x; u) + (a:)uofn.i = Op(<£~" (cr„, 2 VV")) + op (<5„<T«,l) 



(35) 



where sup u is taken over all u such that 

u = uo + (tioi,...,«on) 5 max(|uot|) = o(6 n ), \u \ = 0(5]^ v ). 

i 

In this way we end up with the following corollary. 

Corollary 5.2. Assume that i 7 ^-) is 3 times differentiable with bounded, con- 
tinuous and integrable derivatives. Assume that A can be written as AqI + 
(A i,...,A „) ; where 

A = o P (Sl l ~ v ), maxAoi = o P (6 n ). 



Ther 



sup 



N n (x;A) + fP(x)A { nA =Op{5 1 n - v (o- nfi V^)) + o P {5 n o- n , 1 ). (36) 



Noting that for a £ < 1/2 we have i — e n \ = op{a n ^), we may replace 
i with e n i in the statement of Theorem 3.1. 



5.1.1 Proof of Lemma 5.1 

Let fri,{(-) be an empirical distribution function, associated with £i, . . . , £ n an d 
let /?(') be, respectively, distribution and density function of any of £j. 
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Note that & and r\i are independent for each fixed i, and * f n = f e . Recall 
that £ n>r is defined in the analogous way as £ n y, see (2). From (3) we obtain 
that £„.i = P (a n s). 
Furthermore, let 

n p 



Note that S njP is defined in the same way as S ntP , but we use £,'s in the former 
instead of Si's in the latter. Nevertheless, we conclude from (5) and (6) that for 
a e < 1/2, 

a-^S^x)^ ff\x)Z 2 , (37) 
where Z 2 is the same random variable as in (4). Otherwise, if a e > 1/2, then 



n- 1/2 5„,i(x)^*(x), 



(38) 



where iff is a Gaussian process and the convergence is in the Skorokhod topology. 
We compute 



N n (x;u ) = n J (F n (x + u - v) - F n (x - v)) d(F n ^(v) - F € («)) 
= n J (F n .t:(v) - F^(vj) (f v {x +u -v)- f v (x - vj) dv 
= n J (F n ^(v) - Fe(v) + fz(v)£ n ,i/n) (f n (x + u -v)- f v (x - v)) dv 

- (fe(x + U ) - f e (x)) £„,! 

S n ,l(v) (fr,{x +U0—V)- f n (x - v)) dv - fP (x)wo£n,l + O("o)£n,l 

S n ,i{v)f^(x - v)u (v) dv - / £ (1) (x)u £ n , 1 + 0(«o)Cn,l) 



where uq(v) lies between x — v and x + uq — v. From (5) and (6) we conclude 
that sup^ |<9n,i(«)| = Op(a n ,2 V ^/n). Therefore, with a 1 > v > 0, 



sup sup 

\u \<Sl-" * 



N n (x; u ) + fP(x)uoZ n ,i\ = Op (C(<r n , 2 V y^) + # (1 ~"V„,i) . 



□ 



5.2 Martingale part 

The proofs for martingale part are standard, in particular, they are similar as 
in [4]. However, some details are different, since the main theorems involve 
non-standard scalings rt -1 / 2 and c^ 2 , rather than ct~\. 

Lemma 5.3. Assume that ||/^|joo < 00. 
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(a) Let x r = r-^—^. If a e < 1/2 and (8) holds, then 

supmax|M„(av;u)| = o P (a n ^)- 

u r£Z 

(b) Let x r = f^7= with e > 0. If a e > 1/2 and (11) holds, then 

sup max |M n (x r ; u)| = opfy'ji). 

In 6oi/i cases sup u is taken over all u such that 

u = uo + («oi,.-.,«on)) max(|«oi|) = o(S n ), \u \ = 0(5*~ u ). (39) 
Let 

A„(z; y) = (Me t < y} - F e (y) - (l {ei < x} - F e {x))) . (40) 

The next lemma establishes tightness-like property of the empirical process 
based on i > 1. Note, however, that it cannot be concluded directly from 
the tightness of a~\K n (-), since the different scaling is involved. 

Lemma 5.4. Assume that ||/^||oo < oo. 

• Ifa s < 1/2, then sup| v _ x |< <T -i \A n (x;y)\ = o P {a na ). 

• Ifa s > 1/2, then sup| y _ 2 ,|< en -i/ 2 \A n (x;y)\ = P (en~ 1/2 ). 

Combining Lemmas 5.3 and 5.4 we obtain the following uniform behaviour 
of the martingale part. 

Lemma 5.5. Under the conditions of Lemma 5.3 we have 
supsup|M n (a:;u)| = o P (<r 1h2 ) + P (ey / n). 

u igK 

As in case of Corollary 5.2 we conclude the following corollary. 

Corollary 5.6. Assume that ||/^||oo < oo. Assume that A can be written as 
A 1 + (A i, . . . , A 0n ), where 

A = o P (5l~ u ), max A 0i = o P (S n ) 

i 

and that (8) or (11) holds respectively for a £ < 1/2 or a e > 1/2. Then 

sup \M n (x; A) | = o P {cr rh2 ) + P (e^/n). (41) 
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Proof of Lemma 5. 3. We prove part (a) only. The proof of the other part is 
analogous. Let 

a n ,i{x) — a%{x) := l{x<E t <x+«»} — E[l{ x < ei < x+lli } \Hi-i] , 

so that M n (x,u) — X)"=i a «( x )- We note that {M n (x, u), H n } is a martingale 
array. Thus, by the Rosenthal's inequality 



E|M„(x,u)| 4 <CE 



Furthermore, |a^(x)| < 1, so that 

n n 

E|M„(x,u)| 4 < Cn^E [(E(a l 2 (x)|H l _ 1 )) 2 ] +C^Ea^(x). (42) 



i=l 



i=l 



Note that 

E[a i a (o:)|W i _i]<E[l {(S4 < !e+ | 1H | } |^_ 1 ]-E[l {ei < a _|^| } |%_ 1 ] =: Hf (x) - Hr ( x ) 

and that for each i, H^~(x) and H~ (x) are nondecreasing. 
Introduce a partition R = U re z[x r , av+i). Then 



Ei7+(x r ) = VH+(x r ) • a„, 2 / 1 < a„, 2 E 
EHr( Xr ) = EHr( Xr ) . a n2 f 1 dx > a„, 2 E 

JXr-X 

Thus, for arbitrary M, 

M M 

E [H+(x r ) - H-(x r )] < a n>2 E 



X r+ 1 



H+(x) dx 
H~ (x)dx 



-M 



--M 



= cr„ j2 E 
< o- n2 E 



XM 

X-M 
X M 

X-M 



(H+(x)~H-(x))dx + 



(H+(x)-H7( x ))dx 



H+{x) dx- H-(x) dx 

J X r -i 

I H+{x)dx- I H-{x)dx 

Jxm JX-M-1 



Note that (recall that & = — 



ff+Or) - ^r(x) = F v ( x - e t + - F v ( x - |t 



and 



|^+(x)-^-(x)|<2| Ul |sup/ r) (x) 



fr,(x-£i + y) dy, 
(43) 

(44) 
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Using (43) we obtain 

M 

E[H+(x r )-Hr( Xr )] 



(45) 



r=-M 



< 1 + cr„ i2 E 

< 1 + cr„,2E 



\ui\ />oo 



fr,{x -Zi+y) dy dx 



\Ui\ J CO 



/,,(x + & + y) ete rfy 



ldy 



2 + 2£7„o|«i|' 



= 2 + (T„ :2 E 
Combining (43), (44) and (45), 

M 

^ E [(i/+(xv) - i?r(^)) 2 ] < C|«i| + Ca n>2 ul 

r=-M 

Also, Eof (x) < E[7J+(x r ) - i?~(a; r )]. By Markov inequality and (42), 



(46) 



P 



1 \ 1 1 / " 

max | M „(x r , u )| > 1 < — V EM n 4 ( Ir , u) = — VE V 

' CT "' 2 / a n,2 r °~n,2 r \i=i 

if™ "1 

< C "EE E {(V(a?(xr)\m-i)) 2 ] +C^^Ea?(x r ) 



a, (x r ) 



y n,2 



< — ^ < \uj\ + ftg n , 2 y]u- + n + ov^E 



y n,2 



The bound converges to under the conditions (8) and (39). 



□ 



Proof of Lemma 5.4- Similarly to (32), A n (x;y) is decomposed as M n (x;y) + 
N n (x;y), where M n (x;y) is the martingale part and N n (x;y) is the LRD part. 
We have 

n 

N n (x; y) = J2 (E[l { x<e i<2/ }|Wi-i] - (F e (y) - F £ (x))) < n\\f v + f e \\oo\y - x\. 
i=i 

(47) 

From [19, Lemma 14], sup\ y _ x \ <en -i/z \M n (x;y)\ = Op(en 1 / 2 ). Therefore, the 
case a £ > 1/2 is proven. 

Furthermore, for a e < 1/2, 



sup \M n (x;y)\ < 2 sup 

\y-x\Ka~\ x£R 



Z2 { l {ei<x} ~ E [l{ ei <ai}|Wi-l]) 



= Op((T„ >2 ). 



□ 
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Proof of Lemma 5.5. We start with a £ < 1/2. We can rewrite ai{x) as follows: 
a%{x) = l{ Ei <x+« t } - 1 {e i <x} ~ (F v (x - & + Ui) - F n (x - &)). 

Let x £ [x r ,x r -\-i), since l{ ej < x } an d ^(x) are nondecreasing functions with 
respect to x we have 

0,i(x) < l{ ei <x r+1 +Ui} ~ 1 {e i <x} ~ (F n ( x ~ 6 + «») - F V ( x r+l ~ &)) 

= Oi(x r+ i) + l{ Ei < Xr+1 } - l{ £i <x} + F v (x r+1 - & + Ui) - F„(a; - & + 
Thus, recalling the definition of A n (x;y) given in (40), 

n 

M n (x, u) = M n (a; r ;u) + (l{ ei <* r+1 } ~ F E (x r+1 ) - (l{ £i < x } - F E (x))) 

i=i 

+ X! C^O^+l _ & + U i) ~ ^U 35 _ & + ""»)) 
i=l 

=: M„(x r ; u) + A n (x; av+i) + B„(a:; av+i; u). 

Now, 

sup sup \M n (x; u)| = sup max sup |M n (x;u)| < supmax|M n (z r ;u)| 

u i£l u rGZ xe [ XriX ,. + 1 ) u r 

+ sup j4„(x; av+i) + supmax max B n (a;;x r +i;u). (48) 
l^i-^2|<<r~ 1 2 u r l6 l If ^ 1 ' 

On account on Lemma 5.3, the first term in (48) is op{a n ^). The same holds 
for the second part by Lemma 5.4. For last term we consider Taylor expansion 
for F n : 

F v (x r +i - & + Ui) = F v (x - Ci + Ui) + fn(s)(x r+ i - x), 

where s G [x — £j + Ui, x r+ \ — £j + m). Thus, the bound on B n (x; av+lJ u) is 
independent of u 

n 1 

B n (x;x r+ i;u) = V" fr,(s){x r+1 - x) < nf v (s) = o(er„ j2 ) 

i=i ' 

since n/cr^ 2 — > for a £ < 1/2. Thus, the proof for a e < 1/2 is finished. 

If a £ > 1/2, then with the choice x r = r-^ the first part in (48) is op{-^n) 
and the same holds for the second part by applying Lemma 5.4. The term 
B n (x; av+i; u) is bounded by 

n 

B n (x;x r+1 ;u) = V" f v (s)(x r+1 - x) < nf n (s)—= = 0(ey/n). 

□ 
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5.3 Proofs of Theorems 3.1 and 3.2 

The result of Theorem 3.1 follows from Corollary 5.2 and uniform op(a n ,2) neg- 
ligibility of the martingale part in Lemma 5.5. 

Now, let a e > 1/2. Corollary 5.6 implies that for each r), 6 > we may 
choose e > small enough so that 



P (sup \n- 1/2 M n (x, A)| > 9 ) < 1 - 

/ 



(49) 



Recall (33). This combined with (36) of Corollary 5.2 and (49) yields 
K n (x) = K n (x) + M n (x; A) 

71 / 71 \ 



t=l \i = l 



= K-„(a:) + /«,(*) J> + / B (x) ]T A, - / e (x) - /j^Ao^i 

i=l \ i=l i=l / 

+0 + °p(V^) + Op (6 ). (50) 

Application of (6) yields 

n- 1/2 (^K n (x) + f £ (x)f^e}j ^Wi(x). 

The result of Theorem 3.2 follows. 

5.4 Proof of (28) 
We have 

E[L b {X 1 ,X 2 )} 1 + 9^1 J s 2 K (s)ds J f (2 \v)dv = l + 0{b 2 ). (51) 

Consequently, E[L&(Xi, X 2 )]e = e + Op (b 2 a n ,i/n) = e + op(a n ,i/n). 

Furthermore, since central moments are bounded by ordinary moments, 

Var \^J2 L t( x i' x j)^ = 

O(l) " 1 " 



j=l J.i'=i 
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It is straightforward to verify that for different indices 

ElLbiX^XALbiX^Xj,)] =l + 0(b). 
Combining this with (51) yields 

E[Lt(X i ,X j )L* b (X i ,X j ,)} = o(b). 
Consequently, if (22) holds, then uniformly in i, 

Var f^E^^^j =°{^b)- 1 )+o{bal 1 /n 2 )=o{al l /n 2 ). (52) 
5.5 Proof of (29) 

Recall (25) and (27). Also, recall that once (27) is evaluated, we may replace 
fb(Xi) with f(Xi). Therefore, we have 

n 1 n 

A, = P {nb 2 ) + E[L b (X 1 ,X 2 )]e nA + - ^ L^X^ej, (53) 



3=1 



where Lb(Xj) = bf(X ) ^ b ( Xi — -^J') an< ^ Ll(Xj) is its centered version. 

Now, the variance of the third term in (53) is 



-2 E *% £ i'] E Lt(Xf) 



= h + h + h + h 



3)3' =1 



- 2 E[ £ , £ ,,]^Cov 



n 2 b 2 



^ n n 

v E E ^-^'] E Cov 



MX, - Xj), —^—KbiXe - A» 



1 n n 



3=1 i=l 



1 n n 

¥ E E i £ 3 2 ] E Cov 



3 = 1 



We start with I\. We claim that 



Q(i) ( 

n 2 b 2 



n£ + ^ ] = OK 2 u /n). 

\.i.J,i' different j=£j' ,i=j or j=j' 
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Indeed, let us verify the case when i,j,f arc different. We have (recall (PI)) 
Cov 



K b (Xi - X 3 ), -±—K h (Xi - X. y ) 



f(X 



< E 



1 



lf(Xi 



f(Xt 



-jr—K^u - v)-^—K b {u - v')f(v)f(v') du dv dv' 

1 -K(s)- 7 —K(s')f(u - sb)f(u - s'b) du ds ds' = 0(b 2 ). 



In I2, the term with all indices different vanishes (recall that we work 

under (PI)). The other terms are verified in the similar way as for Ii, by 
computing expected values of products instead of covariances. We obtain: 



J 2 - ^2<Vr 



nb. j=0« 1 /(n6)). 



Similarly, 



n / n \ 

3=1 \i=l / 



Finally, for I4 let us note that with different we obtain 



Cov 



K b {X t - Xj), —L—K h {X v - Xj] 



Thus, 



0(1) 
n 2 b 2 



f(Xr 



h = 



nb 

\i=£i' -i=3 or V—3) 



0(b~ 



o(n). 



From (53), (51) and the above estimates we obtain 

n 

Ai = P (nb 2 ) + £„ a + P (6 2 cr„ 4 ) + Op 



op(\/n). 



If a e < 1/2 and (23) holds, then the above estimate is op(a n ^)- Likewise, if 
a e > 1/2 and (24) holds, then the bound is op(y/n). Thus, (29) is proven. □ 
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